Using gaze tracking to effect player choices in multi player interactive narratives

ABSTRACT

The present disclosure is directed to controlling outcomes in a game that includes multiple different users playing respective roles of specific virtual characters. The multiple different users may be present at a same physical location, or the different users may be located at different physical locations when movement of their eyes is tracked. Here, a user may choose one of a set of provided selections by simply looking at the chosen selection for a period of time or by looking at the chosen selection and performing an action or gesture. This functionality allows multiple different users to control actions performed by different specific characters via an online multiplayer system. Depending on what a first user looks at, that first user or a second user may be provided with a corresponding set of selections or audio/visual content via respective gaming devices operated by the first and the second user.

BACKGROUND OF THE INVENTION 1. Field of Invention

The present invention generally relates to digital interactive content.More specifically, the present invention relates to uses ofgaze-tracking in multiplayer digital games.

2. Description of the Related Art

There are various interactive content titles that include differentpossible narratives that are shaped based on the actions of a user. Forexample, interactive games may allow a user to control a character (oravatar) and provide an input that specifies how the character interactswith objects (including other characters) in a virtual environment. Forexample, a character may be located in a specific location within thevirtual environment and encounter an assailant character. The user mayopt between having the character fight the assailant or fleeing thescene by inputting instructions via a controller (e.g., press X buttonto begin fighting, press O to run away, or other combinations of buttonpresses). The interactive content title may respond to such inputs byupdating a display of the virtual environment to include a fight sceneor a scene in which the character runs away.

The virtual environment may be implemented in virtual reality (VR) andgenerated by a head mounted display (HMD) or one or more projectors.Navigation and character controls within virtual environments may beimplemented using a variety of controllers, including handheldcontrollers, voice- or gesture-based controllers, wearable sensors andsensor-based controls, and other input devices and systems. The varietyof different controllers may be a barrier for newer, more inexperiencedplayers who have not acquired or developed the muscle memory, reflexes,or basic knowledge about how to use each controller effectively.

There is therefore a need in the art for improved systems and methodsthat allow users to control characters and interactions within thevirtual environment in accordance with eye movement.

SUMMARY OF THE CLAIMED INVENTION

Embodiments of the presently claimed invention may relate to a method, anon-transitory computer readable storage medium, and systems thatidentify content to provide to different specific users playing a gamebased on items that those different specific user's look at when thegame is played. In a first embodiment, a method may include identifyingthat a first player is focusing on an object displayed on a display of adevice of the first player. This the identification may have been madeby evaluating received image data of the eyes of the first player afterthe first player has been associated with a first game character thatinteracts with a second character of the game. Next a first indicationmay be sent to a second player device based on the identification thatthe eyes of the first player are focusing on the object displayed on thedisplay of the first player device. The sending of this first indicationmay command the second player device to display a first set ofselections, and then data may be received from the second player devicethat corresponds to a first selection of the first set of selections.This method may also include providing a first set of audio/visual (AV)content to the first player via the first player device based on thereceived data being associated with the first set of AV content. Thisfirst set of AV content may include audio of the second game characterspeaking to the first game character. After the first set of AV contenthas been provided via the first player device, the method may continueby identifying that the first player has made a second selection of asecond set of selections based on the first player focusing on thesecond selection, and the method may also include sending a secondindication to the second player device to command the second playerdevice to provide a second set of AV content to the second player. Thissecond set of AV content may correspond to the second selection.

Method consistent with the present disclosure may be implemented as anon-transitory computer-readable storage medium where a processorexecutes instructions out of a memory. Here again, the method mayinclude identifying that a first player is focusing on an objectdisplayed on a display of a device of the first player. This theidentification may have been made by evaluating received image data ofthe eyes of the first player after the first player has been associatedwith a first game character that interacts with a second character ofthe game. Next a first indication may be sent to a second player devicebased on the identification that the eyes of the first player arefocusing on the object displayed on the display of the first playerdevice. The sending of this first indication may command the secondplayer device to display a first set of selections, and then data may bereceived from the second player device that corresponds to a firstselection of the first set of selections. This method may also includeproviding a first set of audio/visual (AV) content to the first playervia the first player device based on the received data being associatedwith the first set of AV content. This first set of AV content mayinclude audio of the second game character speaking to the first gamecharacter. After the first set of AV content has been provided via thefirst player device, the method may continue by identifying that thefirst player has made a second selection of a second set of selectionsbased on the first player focusing on the second selection, and themethod may also include sending a second indication to the second playerdevice to command the second player device to provide a second set of AVcontent to the second player. This second set of AV content maycorrespond to the second selection.

Systems consistent with the present disclosure may include a firstgaming device and a second gaming device, where each of these respectivegaming devices include a respective memory and a respective processorthat executes instructions stored at the respective memory. Theprocessor of the first player device may execute instructions thatidentify that a first player that is associated with a first gamecharacter has focused on an object displayed on a display of the firstplayer device. The processor of the first player device may then prepareto send a first indication to the second player device based on theidentification that the first player is focusing on the object displayedon the display of the first player device. This first indication may beassociated with a first set of selections. The processor of the secondplayer device may execute instructions that display the first set ofselections on the display of the second player device based on receiptof the first indication. The processor of the second player device maythen prepare to send data to the first player device that identifiesthat a second player has made the first selection of the first set ofselections by focusing on the first selection. This system may thenprovide a first set of audio/visual (AV) content that corresponds to thefirst selection to the first player via a speaker and a display of thefirst player device based on receipt of the data sent from the secondplayer device. Here again, this first set of AV content including audioof the second game character may be provided via the speaker of thefirst player device. The processor of the first player device may thenprepare to send a second indication to the second player device afteridentifying that the first player made a second selection of a secondset of selections by focusing on the second selection. Next, the systemmay provide a second set of AV content to the second player based on thereceipt of second indication sent from the first player device when theprocessor of the second player device executes further instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary network environment in which a systemfor gaze-based controls in multiplayer games may be implemented.

FIG. 2 is a flowchart illustrating an exemplary method for gaze-basedcontrols in multiplayer games.

FIG. 3 is a flowchart illustrating another exemplary method forgaze-based controls in multiplayer games.

FIG. 4 is a flowchart illustrating yet another exemplary method forgaze-based controls in multiplayer games.

FIG. 5 illustrates a computing system that may be used to implement anembodiment of the present invention.

DETAILED DESCRIPTION

The present disclosure is directed to controlling outcomes in a gamethat includes multiple different users (i.e. players) playing respectiveroles of specific game characters. The multiple different users may bepresent at a same physical location or the different users may belocated at different physical locations (or both) when movement of theireyes is tracked. Here, a user may choose one of a set of providedselections by simply looking at the chosen selection for a period oftime or by looking at the chosen selection and performing an action orgesture (e.g. a hand gesture, eye blink, or head movement). Thisfunctionality allows multiple different users to control actionsperformed by different specific characters via an online multiplayersystem. Depending on what a first user looks at, that first user or asecond user may be provided with a corresponding set of selections oraudio/visual content via respective gaming devices operated by the firstand the second user.

When the several players (users) are located at a same physicallocation, each of those several players may interact via one or moregaming devices, one or more sets of electronics, a display, individualhead mounted displays (HMDs), or some combination thereof. Whendifferent players that are located at different physical locations, eachof those players may interact with a respective set of electronics,HMDs, or gaming systems.

Each of the respective players of a game may be given an opportunity toselect a game character that they wish to “play.” Once a particularplayer selects a character, that player will assume the role of thatcharacter when the game is played. For example, a first player mayselect to play the role of a customer, a second player may select toplay the role of a shop owner, and a third play may select to play therole of a wife of the customer. Such a game may also include “non-playercharacters,” where a gaming device plays the role of one or morecharacters automatically.

When these players play the game, an action performed by one player maybe used to identify selections that may be provided to another player.This action may simply be related to the fact that a first player hasfocused on an object displayed on a display. In an example, imagine ascene in a gun store where there are two characters playing a game.These characters may include a customer and a shopkeeper. In oneinstance, the customer may be a “player character” (i.e. a characterthat act according to choices made by a player that has assumed the roleof the character) and the shopkeeper may be a “non-player character.”Here, the customer may be given the option to “buy an item” or“negotiate a better price.” Depending on the player's choice, theshopkeeper would react in a pre-programmed way (e.g. sell the item orgive a discount).

Alternatively the shopkeeper may be controlled by another player who isprovided their own sets of shopkeeper-specific choices. These choicescould be dictated by the choice made by the customer. For example, ifthe customer chooses to “negotiate a better price,” then the playercontrolling the shopkeeper may be presented with their own options:“offer a lower price” or “laugh and refuse.”

FIG. 1 illustrates an exemplary network environment in which a systemfor gaze-based controls in multiplayer games may be implemented. Asillustrated, the network environment may include different gamingdevices 120/150/170 in communication over a communication network (e.g.,cloud/Internet 110). Each of the gaming devices 120/150/170 may beassociated with one or more players in the same or different physical,geographic locations. FIG. 1 further illustrates that the gaming devices120/150/170 may further be communication with one or more head mounteddisplays 130/140/160 or a game controller 180. Gaming device 120, 150,and 170 may be a gaming system (e.g., Sony™ PlayStation), a computer, ahandheld device, or another computing device.

As illustrated, player 1 head mounted display (HMD) 130 and player 4 HMD140 may communicate with the first gaming device 120. Here, player 1 andplayer 4 may be located in close proximity to the first gaming device120 with which their respective HMDs 130/140 may be communicating. Eachof the different gaming devices (120, 150, & 170) of FIG. 1 may belocated at a different physical location and each of these differentgaming devices may communicate with each other over the cloud orInternet.

FIG. 1 also illustrates a player 2 HMD 160 communicating with the secondgaming device 150 and illustrates a player 3 game controllercommunicating with the third gaming device 170. The different gamingdevices of FIG. 1 may be configured to associate a particular gamecharacter with a particular user or player of the game. When headmounted displays are used, a player may experience the game in a virtualreality environment and depending on the capabilities of a particularHMD a user may provide commands to a gaming device via that particularHMD. When a gaming controller is used, a player (e.g. player 3 of FIG. 1) may view the game on a standard display (e.g. a flat panel monitor)and that player may provide commands to the game using their gamecontroller.

Capabilities of particular head mounted displays may allow a user toprovide control inputs to control actions performed by a game characterthat is associated with a particular player HMD. An HMD may receiveinputs based on eye tracking data, on user gestures (e.g. hand motionsor eye blinking), or combination of both eye tracking and user gestures.In some implementations, the eye tracking and gesture data may also beobtained from cameras and other sensors in a surrounding real-worldenvironment of the respective user. Commands for performing specifictypes of actions in a virtual environment may be identified based onanalysis of where a user is gazing. For example, a particular action maybe directed in towards gaze direction that is held for a thresholdperiod of time.

Apparatus of the present disclosure may also include multiple speakersthat provide direction-based noises. Here, sounds may be provided to auser via a speaker located at the left, the right, in front of, behind,above, or below a person. Such a three-dimensional (3D) speaker systemmay be incorporated into a HMD or may be setup in a room of a gameplayer. When such a 3D speaker system is used, noises provided to a usermay be directional. For example, the ringing of a bell to the player'sleft, a rattle snake rattling at the player's feet, an airplane flyingabove the player, or the sound of a gun cocking behind the player. Whilesuch 3D speaker systems may be supported by a particular game, HMD,speaker system, and/or gaming system, the game may still function evenwhen certain players do not have a 3D speaker system available. A gamecontroller, such as the player 3 game controller 180 may be any type ofcontroller known in the art (e.g. a joystick, a two-handed controller,or one or more one handed controllers). Even when a game controller isused, a gaming device or the controller itself may include camerasdedicated to track eye motion of users.

Methods of the present disclosure may provide user's with various typesof realism based on limitations of the types of devices that are used toreceive user input and to provide content (e.g. sounds, images, orvideo) to users. While FIG. 1 illustrates head mounted displays, gamecontrollers and gaming devices as separate physical devices, thefunctionality of each of these devices may be incorporated into a singledevice. For example, a head mounted display (HMD) may also perform thefunctions of a gaming device. In such an instance, two HMDs located at asame physical location may act as separate gaming devices or one ofthese HMDs may act as a master HMD that receives inputs from the secondHMD.

In certain instances, graphics or audio may be generated at a cloudserver or cloud computing device, which may send generated graphicalimages or audio content to specific user devices. This could allowcomputing devices with limited graphics generating capabilities toreceive and display high quality graphical images. Moreover, many of thefunctions described herein may be performed in the cloud by one or morecloud servers or cloud computing device in accordance with cloud-basedservices associated with a current session. For example, associated dataand content may be stored in one or more cloud storage devices, andcloud computing resources may be used to process such data inconjunction with data received from one or more user devices (e.g.,devices 120/150/170) in a current session.

FIG. 2 is a flowchart illustrating an exemplary method for gaze-basedcontrols in multiplayer games. Such a method may be performed by agaming device used for multiplayer gameplay. Here, the first and secondplayer may be located at different locations and game actions orsequences may be controlled using communications that are sent torespective gaming devices via the Internet (e.g. the cloud or Internet110) of FIG. 1 .

As mentioned previously, the scenes described above may take place in avirtual reality (VR) setting where players wear head mounted displays(HMDs). In an instance when the player controlling the shopkeeper haschosen to make the shopkeeper a woman outfitted in a low-cut blouse andwhen the player controlling the customer is looking at the shopkeeperfrom a first-person perspective, the shopkeeper player may be provided afirst set of optional selections when the customer focuses on theshopkeepers face and may be provided a second set of optional selectionswhen the customer focuses on the shopkeepers low-cut blouse. In thefirst instance, when the gaze of the customer player remains on theshopkeeper's face (e.g. maintaining “eye contact”) while choosing tonegotiate a better price, then the shopkeeper's response selectionchoices may be to “offer a lower price” or “laugh and refuse.”

In the second instance, when the customer player's gaze drifts to theshopkeeper's low-cut blouse (staring at her cleavage) while choosing tonegotiate a better price, the difference in gaze may result in theshopkeeper's being offered choices to say “My eyes are up here” or tosmile and flirt by saying “1 see that you like what you see.” In otherwords, the shopkeeper's choices may be dictated by the gaze orfocal/focus point of the customer player.

The method of FIG. 2 begins with step 210, where a gaze direction or eyefocal/focus point of a player (e.g. the first or the second player) maybe identified. As discussed above, this could include identifyingwhether the customer player (first player) is looking at theshopkeeper's face or at the shopkeeper's cleavage and the shopkeeperplayer (second player) could be provided with the corresponding set ofplayer selections discussed above. After determination step 210identifies where the first player (the customer player) is looking,determination step 220 may identify whether the second player (theshopkeeper player) should be provided with a set of selections.Determination step 220 may identify that the second (shopkeeper) playershould not be provided with a set of selections because the first(customer) player is not looking at the face or the cleavage of theshopkeeper. In such an instance, program flow may move to determinationstep 250, discussed below.

When determination step 220 identifies that the second (shopkeeper)player should be provided with selections, an appropriate set ofselections could be provided to the second (shopkeeper) player. Thiscould include displaying text in a display viewed by the second playerand the second player could select a response from a set of responses.Here, selections could be made by the second (shopkeeper) player bystaring at a particular line of text for a threshold amount of time(e.g. 5 seconds) or could include the second (shopkeeper) player staringat the particular line of text and making a gesture (e.g. a hand motion,eye motion, or eye blinking). This may include identifying a gazedirections or focal/focus point of the second player in step 230 of FIG.2 after which content may be provided to the first (customer) playerthat corresponds to the selection made by the second (shopkeeper)player. Step 240 may include playing a video of the shopkeeper characterlaughing and saying “no, my price is firm, no discount.”

After step 240, program flow may move to determination step 250 thatidentifies whether the first (customer) player should be offered a setof response selections. For example, these selections could include“alright I'll pay your price” or “I can only spend $200 and am willingto trade this knife to cover the difference.”

As discussed above, determination step 250 may be performed afterdetermination step 220 identifies that the second (shopkeeper) playershould not be provided selections. In such an instance, the first playermay be provided with selections of saying “please show me that fortycaliber semi-auto pistol” or may include “what pieces do you have that Ican buy for $200.”

The first (customer) player may be provided with a set of selections anda selection may be identified based on the gaze direction or focal/focuspoint of the first player in step 260 of FIG. 2 . After step 260,content may be provided to the second (shopkeeper) in step 270 of FIG. 7. This content may include audio of the first (customer) player stating“what pieces do you have that I can buy for $200.

Alternatively, when determination step 250 identifies that the firstplayer selections should not be provided to the first player, programflow may move back to step 210 where a gaze direction or focal point ofthe first and/or the second player may be identified.

After step 270, program flow may move to determination step 280 thatidentifies whether a third character should participate in a next sceneof interactions between the first and second players, when yes programflow may move to step 290 where a third character is introduced. Here,this third player or character could be controlled by a third player oractions performed by the third character could be automated by a gamingdevice selecting actions that the third character should perform. Suchan automated character may be referred to as a non-player characterbecause the actions of that character are not controlled by a player.

When determination step 280 identifies that a third character should notparticipate in the next scene of the game, program flow may move back tostep 210 where a gaze direction or focal point of the first and/or thesecond user may be identified. This may result in yet another sequenceof interactions between the first player controlling actions of thecustomer character and the second player controlling actions of theshopkeeper player. Depending on where each respective player is looking,different selections may be provided to respective players andadditional content may be provided based on selections made by thoserespective players. Such interactions could continue until the customermakes a purchase or leaves the gun store, for example.

FIG. 3 is a flowchart illustrating another exemplary method forgaze-based controls in multiplayer games. Such a method may be performedby the game device when a third character or player participates in thegame session. The steps of FIG. 3 may be performed after determinationstep 280 of FIG. 2 identifies that a third character should participatein the game. FIG. 3 begins with determination step 310 that identifieswhether a third player is present to control actions of the thirdcharacter. When determination step 310 identifies that a third player isnot present, program flow may move to step 340 where an action isperformed by a gaming device that automatically selects an action thatthe third character should perform.

When determination step 310 identifies that a third player is present,program flow may move to step 320 where a gaze direction or focal/focuspoint of any or all of the players may be identified. Next, a player maybe identified to which a set of selection options could be provided.Determination step 330 may then identify whether an identified playershould be provided with a particular set of selection options. Heraagain, different sets of selection options provided to specific playerscould be a function of locations where specific players are looking.Determination step 330 may be performed either after step 320 or afterstep 340 depending on whether a third player is available to play thegame. When determination step 330 identifies that a selection should notbe provided, program flow may move back to step 320 where the gazedirection of the players is evaluated again. This may result in programflow looping between steps 320 and 330 repetitively until determinationstep 330 identifies a player that should be provided with a set ofselection options.

When determination step 330 identifies a player that should be providedwith selection options, the set of selection options that corresponds toa gaze direction of one player may be provided to any player of thegame. In certain instances, instead of a gaze direction of the firstplayer resulting in the selection options being sent to the secondplayer, the selection options may be sent to the first player. Forexample, the first player may look at a box of ammunition and the first(customer) player may be provided with a set of different statements(such as comments or questions) that the first (customer) player couldask the second (shopkeeper) player. Such a comment could be “I like thatplus P hollow point ammo” or question could be “what is the price ofthat box of ammo?”

Next in step 350, an identified set of selection options could beprovided to the identified player and then in step 360 content could beprovided that corresponds to a selection made by the identified player.This could result in the first (customer) player asking the second(shopkeeper) player “what is the price of that box of ammo?”

After step 360 program flow may move to determination step 370 where anon-player character action may be performed, when yes, program flow maymove to step 340 where a non-player character action is performed by thethird character. This could include the third charter making a statementselected automatically by a gaming device. When determination step 370identifies that a non-player character action should not be performed,program flow may move back to step 320 where a gaze direction orfocal/focus point of one or more player is identified again.

The steps of FIG. 3 illustrate that actions performed by multiplecharacters may be performed based on gaze direction or focal/focuspoints of specific users playing a game. This may be true even when somecharacters actions are automated. The steps of FIG. 3 may also beperformed based on the steps of FIG. 2 being performed.

In yet another example relating to the shopkeeper scene, when thecustomer is facing the shopkeeper behind the counter, a bell rings tothe left of the customer (discernable via 3D audio) as the door to theshop opens and a third character enters. In an instance when the ringingof the bell is related to the customer's wife (e.g. thirdplayer/character of FIG. 3 ) entering the shop, different selectionoptions may be provided to specific players based on the gaze directionof the customer player.

In an instance when the customer turns and his gaze falls upon his wife,then selection options that may be provided the customer player mayinclude “What are you doing here?” and “You're late.” The shopkeeper, inthe meantime, may become passive, allowing for words to be exchangedbetween the customer and his wife.

In an instance when the customer doesn't turn towards the ringing bell,then his wife may enter out of his field of view. The scene between thecustomer and the shopkeeper may then continue as before, with thecustomer trying to negotiate a better price and the shopkeeper's makingselections as discussed above. In the meantime, a player playing thewife may observe the scene from a different perspective including herhusband (the customer) and the shopkeeper and a direction of the gamemay be based on a gaze direction of the wife.

In an instance when the customer doesn't realize his wife has enteredthe store, that information can be used to change the shopkeeper'sreaction to the customer looking at the second (the shopkeeper) player'scleavage. Now, the second (shopkeeper) player's choices may include arecitation to the customer character of “Isn't that your wife?” or mayinclude a statement to the wife “Your husband should keep his eyes tohimself.”

In an instance when the third (wife) player isn't looking at the secondplayer (customer, her husband) at the time when the second (customer)player is inspecting the shopkeeper's cleavage, the third (wife) playermay be provided with other selection options. Here, the third (wife)player may be provided with selection options of sayings that include“How dare you accuse my husband of such a thing?” Alternatively, whenthe third (wife) player's gaze indicates that she saw that her husband(first player) did in fact stare at the shopkeeper's (second player's)cleavage, then the third (wife) player may make a selection to say toher husband (first player), “I knew you were a creep, Larry.”

In another example, a first player may be playing the role of adetective character investigating a crime scene in the presence of asuspect character controlled by a suspect player. Here, both actionsperformed by the detective player and the suspect player may becontrolled by actions performed by different players. In an instancewhen the gaze of the detective player falls upon a piece of keyevidence—say, a murder weapon—the suspect player might be presented withchoices like “attack the detective” or “flee the scene.” In an instancewhen the detective's gaze never touches the evidence, the suspect playermay receive no choices. In each of these scenarios, actions performed bycertain players may result in selection options being provided to anyother player, where each action and a resulting selection cause the gameto move in a different direction in ways that may result in differentoutcomes.

In certain instances a the gaze of a player may triggers an action to beperformed by a non-player character (NPC): A customer player's gaze on aprice tag may be used to trigger an NPC shopkeeper to automatically say“May I help you find something?” In an instance when the player does notlook at the price, the NPC shopkeeper may not say the “May I help youfind something?” question. Instead, other statements made by the NPCshopkeeper may be provided when the customer looks at another item.

The duration of a gaze may be used to change an intensity of reaction. Aquick glance at the shopkeeper's cleavage may provoke one set of choices(or none at all), whereas a prolonged gaze might provoke a different setof more aggressive choices. Gaze threshold times may correspond to anumber of seconds, for example two seconds, five seconds, or sevenseconds.

FIG. 4 is a flowchart illustrating yet another exemplary method forgaze-based controls in multiplayer games. Such a method may be performedby a game device to identify a perspective or an emotion of a player.FIG. 4 begins with step 410 where player pupil dilation data may beevaluated. This may include collecting first set of eye image data ofthe player at a first point in time, evaluating that eye image data toidentify how dilated the eyes of the player are at the first point intime, collecting a second set of image data at a second point in time,and identifying how dilated the eyes of the player are at the secondpoint in time. Determination step 420 may then identify whether theplayer's eye dilation has changed. When the player's eye dilation hasnot changed, program flow may move to step 450 where facial expressiondata of the player is evaluated.

When determination step 420 identifies that the players eye dilation haschanged, program flow may move to step 430 where a perspective of theplayer may be identified. Changes in player eye dilation may be used toidentify perspectives of “interest” or “suspicion,” for example. Whenthe player's pupil widens (gets larger), a gaming system may identifythat the player is “interested” in an object or character that theplayer is currently looking at. In an instance when the player's pupilnarrows (gets smaller), the gaming system may identify that the playeris “suspicious” of something that they are looking at. Because of this,methods of the present disclosure may make judgements based on observedchanges in pupil dilation of a player. When a player appears to beinterested in an object, that player may be provided with content (audioand/or visual content) associated with the interest in step 440. Forexample, when the player's pupil increases in size when viewing a car,that player may be provided with a set of selection options or with anaudio question asking if the player would like to drive the car.

In an instance, when the pupil of the player narrows, another charactermay provide an audio message stating “don't worry, you can trust me” instep 440 of FIG. 4 based on the gaming system identifying that theplayer appears to be suspicious because of narrowing of the player'spupils.

After step 440, program flow may move to step 450 where facialexpression data of a player may be evaluated. Here, interpreted player'sfacial expressions and/or emotions may be used to change reactions ofspecific characters. Here, either a full-face camera or special HMDcamera may be used to collect images such that the player's expressionand/or emotion may be detected. The evaluation performed in step 450 mayinclude collecting multiple images of a player's face overtime and mayinclude identifying that a facial expression of the player has changed.Alternatively, a facial expression from a single image may be evaluatedto see if that single facial expression corresponds to an emotion.Determination step 460 may then identify whether a current facialexpression or a change in facial expression likely corresponds to anemotion. When determination step 460 identifies that the facialexpression data does not correspond to an emotion, program flow may moveto step 490 where other actions may be performed.

When determination step 460 identifies that the facial expression datacorresponds to an emotion, program flow may move to step 470, where aspecific emotion (e.g. confused, angry, happy, excited, or nervous) maybe identified. For example, a confused emotion may be identified by aslight squinting of the eyes of the player. The emotion of anger may beidentified based on a more pronounced squinting of the eyes, wrinklesforming between the eyebrows of the player, by a grimace of the player'smouth, or some combination of these factors. The emotion of happy may beidentified based on identifying that the player is smiling and theemotion of excited may be identified based on identifying that anopening of the player's eye lids have increased in size. The emotion ofnervousness may be identified by a smile and a cackling laugh or bynoticing that a player is moving their body in a jerky way.

Next in step, 480 of FIG. 4 an action may be initiated based on theidentified emotion. These expressions or detected emotions may be usedas information may be used that to change a narrative of the game. Forexample, in an instance when the player's face indicates confusion, thenthe shopkeeper may be given additional explanatory lines of dialogueabout the product. In an instance when the player's expression indicateshappiness, then the shopkeeper's reaction may be more pleasant than ifthe player looks angry. Here again, the shopkeeper's statements may bebased on user selection options by a player playing the role of theshopkeeper or may be made by a gaming device may be programmed toautomatically provide statements that correspond to a player'sexpression or identified emotional state. After step 480, program flowmay move to step 490 where other actions are performed. These otheractions may include performing the steps discussed in respect to FIG. 2or 3 .

FIG. 5 illustrates a computing system that may be used to implement anembodiment of the present invention. The computing system 500 of FIG. 5includes one or more processors 510 and main memory 520. Main memory 520stores, in part, instructions and data for execution by processor 510.Main memory 520 can store the executable code when in operation. Thesystem 500 of FIG. 5 further includes a mass storage device 530,portable storage medium drive(s) 540, output devices 550, user inputdevices 560, a graphics display 570, peripheral devices 580, and networkinterface 595.

The components shown in FIG. 5 are depicted as being connected via asingle bus 590. However, the components may be connected through one ormore data transport means. For example, processor unit 510 and mainmemory 520 may be connected via a local microprocessor bus, and the massstorage device 530, peripheral device(s) 580, portable storage device540, and display system 570 may be connected via one or moreinput/output (I/O) buses.

Mass storage device 530, which may be implemented with a magnetic diskdrive or an optical disk drive, is a non-volatile storage device forstoring data and instructions for use by processor unit 510. Massstorage device 530 can store the system software for implementingembodiments of the present invention for purposes of loading thatsoftware into main memory 520.

Portable storage device 540 operates in conjunction with a portablenon-volatile storage medium, such as a FLASH memory, compact disk orDigital video disc, to input and output data and code to and from thecomputer system 500 of FIG. 5 . The system software for implementingembodiments of the present invention may be stored on such a portablemedium and input to the computer system 500 via the portable storagedevice 540.

Input devices 560 provide a portion of a user interface. Input devices560 may include an alpha-numeric keypad, such as a keyboard, forinputting alpha-numeric and other information, or a pointing device,such as a mouse, a trackball, stylus, or cursor direction keys.Additionally, the system 500 as shown in FIG. 5 includes output devices550. Examples of suitable output devices include speakers, printers,network interfaces, and monitors.

Display system 570 may include a liquid crystal display (LCD), a plasmadisplay, an organic light-emitting diode (OLED) display, an electronicink display, a projector-based display, a holographic display, oranother suitable display device. Display system 570 receives textual andgraphical information, and processes the information for output to thedisplay device. The display system 570 may include multiple-touchtouchscreen input capabilities, such as capacitive touch detection,resistive touch detection, surface acoustic wave touch detection, orinfrared touch detection. Such touchscreen input capabilities may or maynot allow for variable pressure or force detection.

Peripherals 580 may include any type of computer support device to addadditional functionality to the computer system. For example, peripheraldevice(s) 580 may include a modem or a router.

Network interface 595 may include any form of computer interface of acomputer, whether that be a wired network or a wireless interface. Assuch, network interface 595 may be an Ethernet network interface, aBlueTooth™ wireless interface, an 802.11 interface, or a cellular phoneinterface.

The components contained in the computer system 500 of FIG. 5 are thosetypically found in computer systems that may be suitable for use withembodiments of the present invention and are intended to represent abroad category of such computer components that are well known in theart. Thus, the computer system 500 of FIG. 5 can be a personal computer,a hand held computing device, a telephone (“smart” or otherwise), amobile computing device, a workstation, a server (on a server rack orotherwise), a minicomputer, a mainframe computer, a tablet computingdevice, a wearable device (such as a watch, a ring, a pair of glasses,or another type of jewelry/clothing/accessory), a video game console(portable or otherwise), an e-book reader, a media player device(portable or otherwise), a vehicle-based computer, some combinationthereof, or any other computing device. The computer can also includedifferent bus configurations, networked platforms, multi-processorplatforms, etc. The computer system 500 may in some cases be a virtualcomputer system executed by another computer system. Various operatingsystems can be used including Unix, Linux, Windows, Macintosh OS, PalmOS, Android, iOS, and other suitable operating systems.

The present invention may be implemented in an application that may beoperable using a variety of devices. Non-transitory computer-readablestorage media refer to any medium or media that participate in providinginstructions to a central processing unit (CPU) for execution. Suchmedia can take many forms, including, but not limited to, non-volatileand volatile media such as optical or magnetic disks and dynamic memory,respectively. Common forms of non-transitory computer-readable mediainclude, for example, a FLASH memory/disk, a hard disk, magnetic tape,any other magnetic medium, a CD-ROM disk, digital video disk (DVD), anyother optical medium, RAM, PROM, EPROM, a FLASH EPROM, and any othermemory chip or cartridge.

While various flow diagrams provided and described above may show aparticular order of operations performed by certain embodiments of theinvention, it should be understood that such order is exemplary (e.g.,alternative embodiments can perform the operations in a different order,combine certain operations, overlap certain operations, etc.).

What is claimed is:
 1. A method for providing gaze-based controls, themethod comprising: identifying that a gaze of a first player iscurrently focused on a virtual object in a virtual environment based onimage data of one or more eyes of the first player after a firstcharacter associated with the first player interacts with a secondcharacter in the virtual environment; sending a first command to asecond user device based on the identified gaze being focused on thevirtual object, wherein the command is executable by the second userdevice to display a first set of options, and data is received from thesecond user device that corresponds to a selection from the first set ofoptions; providing a first set of audio-visual (AV) content to the firstuser device based on the selection from the first set of options, thefirst set of AV content including audio of the second character speakingwithin the virtual environment; identifying that the first player hasmade a selection from a second set of options based on an updatedlocation of the identified gaze within the virtual environment; andsending a second command to the second user device, wherein the secondcommand is executable by the second user device to display a second setof AV content corresponding to the selection from the second set ofoptions.
 2. The method of claim 1, wherein the audio of the secondcharacter speaking includes a set of words that are part of aconversation between the first character and the second character. 3.The method of claim 1, wherein the second set of AV content includes aset of words spoken by the first character that are part of aconversation between the first character and the second character. 4.The method of claim 1, wherein at least one of the first set of AVcontent and the second set of AV content includes three-dimensional (3D)noise initially associated with a direction.
 5. The method of claim 1,further comprising: identifying that the identified gaze is moving in adirection within the virtual environment; and generating a display of animage of a third character based on the movement of the identified gazein the identified direction.
 6. The method of claim 5, furthercomprising: receiving data from a third user device associated with thethird character; and providing a third set of AV content to the firstuser device based on the data received from the third user device. 7.The method of claim 1, further comprising tracking the identified gazeof the first player to the updated location within the virtualenvironment, wherein text associated with the selection from the secondset of options is displayed at the updated location, wherein theselection from the second set of options is identified based on theidentified gaze being directed to the text at the updated location. 8.The method of claim 7, further comprising identifying that theidentified gaze is focused on the text for at least a predeterminedperiod of time.
 9. The method of claim 1, wherein identifying that thefirst player has made the selection from the second set of options isfurther based on a detected gesture by the first player.
 10. The methodof claim 9, wherein the gesture includes blinking of the eyes of thefirst player.
 11. The method of claim 10, wherein the blinking includesat least two distinct eye closings and eye openings.
 12. The method ofclaim 1, further comprising: identifying that a size of a pupil of atleast one of the first player or the second player has changed; andperforming an action based on the identified change in pupil size. 13.The method of claim 1, wherein identifying that the first player hasmade the selection from the second set of options is further based on anexpression of the first player.
 14. The method of claim 13, furthercomprising matching the expression of the first player to one or moreindicators of one or more predetermined emotions.
 15. The method ofclaim 14, wherein the second set of option is further based on one ofthe predetermined emotions that matches the expression of the firstplayer.
 16. A non-transitory, computer-readable storage medium havingembodied thereon a program executable by a processor for implementing amethod for providing gaze-based controls, the method comprising:identifying that a gaze of a first player is currently focused on avirtual object in a virtual environment based on image data of one ormore eyes of the first player after a first character associated withthe first player interacts with a second character in the virtualenvironment; sending a first command to a second user device based onthe identified gaze being focused on the virtual object, wherein thecommand is executable by the second user device to display a first setof options, and data is received from the second user device thatcorresponds to a selection from the first set of options; providing afirst set of audio-visual (AV) content to the first user device based onthe selection from the first set of options, the first set of AV contentincluding audio of the second character speaking within the virtualenvironment; identifying that the first player has made a selection froma second set of options based on an updated location of the identifiedgaze within the virtual environment; and sending a second command to thesecond user device, wherein the second command is executable by thesecond user device to display a second set of AV content correspondingto the selection from the second set of options.
 17. A system forproviding gaze-based controls, the system comprising: memory that storesinformation regarding audio-visual (AV) content associated with avirtual environment; a communication interface that communicates over acommunication network with at least a first user device and a seconduser device, wherein the communication interface: receives data from oneor more eye-tracking sensors associated with the first user device,wherein the data from the eye-tracking sensors indicate that a gaze of afirst player is currently focused on a virtual object in the virtualenvironment based on image data of one or more eyes of the first playerafter a first character associated with the first player interacts witha second character in the virtual environment, sends a first command toa second user device based on the gaze being focused on the virtualobject, wherein the command is executable by the second user device todisplay a first set of options, and data is received from the seconduser device that corresponds to a selection from the first set ofoptions, provides a first set of the AV content to the first user devicebased on the selection from the first set of options, the first set ofthe AV content including audio of the second character speaking withinthe virtual environment; and a processor that executes instructionsstored in memory, wherein the processor executes the instructions to:identify that the gaze of the first player is at an updated locationwithin the virtual environment, and identify that the first player hasmade a selection from a second set of options based on the updatedlocation of the identified gaze within the virtual environment, whereinthe communication interface sends a second command to the second userdevice, and wherein the second command is executable by the second userdevice to display a second set of the AV content corresponding to theselection from the second set of options.
 18. The system of claim 17,wherein the memory includes cloud storage, and wherein the first set ofthe AV content and the second set of the AV content are provided fromthe cloud storage.